Despite Sweeping Scandals, Big States
Don't Check for Cheating by Teachers
by Lois Beckett | @loisbeckett
ProPublica
http://www.propublica.org/article/despite-sweeping-scandals-big-states-dont-check-for-cheating-by-teachers
This summer, as students enjoyed their summer vacations,
education officials in many states were busy handling sweeping investigations [1]
into teacher cheating [2].
In one school district in Atlanta,
at least 178 teachers and principals were implicated in a widespread falsification of student test scores [3]. They had taken students' standardized test
sheets, erased wrong answers and replaced them with the right ones. One teacher
told investigators that the district was "run like the mob" and that
she was afraid of retaliation if she didn't participate.
The cheating in Atlanta
was uncovered in part thanks to two simple checks that states can conduct to
look for suspicious test results.
The same machines that grade the penciled-in bubbles of
standardized tests can also tally how many answers on the tests have been
erased and changed from wrong to right. The technique, called "erasure analysis,"
flags suspicious patterns of answers that may indicate teachers have tampered
with answer sheets to inflate their students' scores.
In Atlanta,
students' test scores also jumped or dropped from year to year in unlikely
ways. Checking for such dramatic swings is another way of
spotting potential teacher cheating.
Experts view such screening as crucial. Teachers and
principals are faced with increasing incentives to cheat, since student scores
are being used to determine whether schools get funding and how teachers and
principals get paid. U.S. Secretary of Education Arne Duncan recently told USA Today that
states should require such screening [4].
Some states are indeed becoming more aggressive. Florida and Illinois
instituted more rigorous screenings of statewide tests this year. (Yesterday, USA Today noted that 20 states regularly screen [4].)
Yet, some of the largest states have lagged.
California stands out. The state’s Department of Education conducted erasure analysis
on tests for several years but ended the program in 2009. John Boivin, an administrator in the department, said screening
was stopped as part of massive cuts after a drastic budget shortfall two years
ago. (Boivin also said the erasure analysis cost the
state $105,000 per year.)
Deborah Sigman, California's
deputy superintendent of public instruction, said the state ultimately had to
choose between eliminating some of the tests themselves and scaling back test
oversight.
Sigman also said the state has always uncovered more cheating via on-the-ground
reports, which the state now relies on. The number of reports of teacher
cheating has increased from 69 three years ago to 263 in the past year, she
said.
But Boston
College Professor Walt
Haney, an expert on testing, said screening is critical.
"Any large districts or state that didn’t employ
those techniques would have its head in the sand," Haney said. "Given
the number of large cheating scandals that have emerged over the last 20 years,
any large institution would be derelict in not instituting some of the widely
documented techniques for identifying cheating."
Sigman said California
plans to reinstate erasure analysis as soon as it has the money to do
so—perhaps this spring.
"This is a real priority," she said. And, she
noted, "It's kind of a small investment."
Other states have also been slow to act. In Pennsylvania and New
Jersey, officials screened for suspicious levels of
erasures, only to let results gather dust for years until local journalists
investigated and published the results themselves.
"People don’t want to know," said Jennifer
Jennings, a sociology professor at New
York University
who specializes in education. "People would rather hold their noses and
hope the scores mean what they think they mean. At every level of the system,
there are adults who have an interest in the scores going up."
Such methods as erasure analysis don’t provide proof on
their own that cheating has occurred. Because the screenings depend on
statistical analysis, they can only say which results are typical and which
ones are highly improbable.
When USA
Today consulted statisticians about the 2009 test
results from one elementary school in Washington,
D.C., they were told that so many
erasures by chance were less likely than winning the Powerball
grand prize with a $1 lottery ticket [5].
Screening methods do flag schools that are later cleared
on further investigation, but screening also consistently identifies cheating
that might otherwise have gone undiscovered.
In New Jersey,
the state Department of Education has conducted erasure analysis of state tests
since 2008. But the department did not investigate any of the schools flagged
in those reports until this summer, when the Asbury Park Press successfully
sued the department to obtain a copy of the analysis and made
it public [6]. (Spokesman Justin Barra told ProPublica that while
the department did not share the reports with districts, the state did use them
to decide whether to send observers on test days.)
Following the Asbury Park Press expose
[7], 34 schools are now under investigation.
Pennsylvania's Department of Education received an erasure analysis report in 2009 that
flagged dozens of schools for potential cheating but left the results untouched
for two years and did not notify school districts about the anomalies [8]. After reporters for an education blog obtained the report and made it public [1], the state ordered initial investigations of 89
schools.
In Texas,
assessment officials said the state's education agency has done erasure
analysis reports for at least 10 years. But until this year, the state had not
used them to investigate schools unless school staff members or test monitors
also had submitted cheating complaints.
Texas' policy continued despite a 2007 update to the state's education code
requiring the adoption of statistical measures to screen for cheating and a
procedure for investigating schools with suspicious results. The state will
finally implement those measures at the end of the school year.
Criss Cloudt, Texas' associate commissioner of
assessment and accountability, said the state has relied on its rigorous test security
measures, including seating charts, honor pledges and legally binding oaths
that test administrators must sign, to prevent cheating.
Testing experts say that test security, while important,
is not a substitute for screening measures.
Other states are also beginning more rigorous screenings.
In a report released last week, New York's Department of Education
recommended a multipart screening of state tests, looking for score jumps,
unlikely patterns of answers, and high levels of erasures. North Carolina also is considering more
regular erasure analysis.
Florida has screened a subset of high-stakes tests since 2004 and implemented a
state-of-the-art analysis of all statewide test results this year.
"Generally, from leadership we have gotten a good
but nervous response. This obviously is not the kind of work that people relish
doing," said Kris Ellington, Florida's
deputy commissioner for accountability, research and measurement.
"Other than that, the discomfort
with it, there aren’t any drawbacks."
Of course, instituting a more rigorous screening of test
results means facing how widespread teacher cheating actually is. The new
analysis revealed more suspicious results from both students and teachers than
in previous years, Ellington said.
But Ellington said the problematic results represented a
tiny fraction of the tests administered—and that screening has given Florida confidence that
the remainder of its scores are valid.
"We want to make sure that no corruption is part of
this process," she said. "We don’t believe that there are big pockets
of problems. But we can’t just live in our happy place and believe it. We have
to know it."